Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci(checks): build cppcheck from sources #180

Merged
merged 1 commit into from
Jun 4, 2023
Merged

ci(checks): build cppcheck from sources #180

merged 1 commit into from
Jun 4, 2023

Conversation

P403n1x87
Copy link
Owner

@P403n1x87 P403n1x87 commented Jun 4, 2023 •
edited
Loading

The version of cppcheck available from the package manager tends to be outdated. In this change we build from sources instead. We cache the build result for faster execution.

@P403n1x87
ci(checks): build cppcheck from sources
57f360b 
The version of cppcheck available from the package manager tends to be
outdated. In this change we build from sources instead. We cache the
build result for faster execution.
@P403n1x87 P403n1x87 added the ci/cd label Jun 4, 2023
@P403n1x87 P403n1x87 marked this pull request as ready for review June 4, 2023 12:06
@codecov
Copy link

codecov bot commented Jun 4, 2023

Codecov Report

Patch coverage has no change and project coverage change: +0.16 🎉

Comparison is base (369b825) 69.96% compared to head (57f360b) 70.12%.

Additional details and impacted files 
@@            Coverage Diff             @@
##            devel     #180      +/-   ##
==========================================
+ Coverage   69.96%   70.12%   +0.16%     
==========================================
  Files          25       25              
  Lines        2474     2474              
  Branches      730      730              
==========================================
+ Hits         1731     1735       +4     
+ Misses        412      410       -2     
+ Partials      331      329       -2

see 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@github-actions
Copy link

github-actions bot commented Jun 4, 2023

Austin Benchmarks

Running Austin benchmarks with Python 3.10.11

Wall time [sampling interval: 1]

Wall time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 109000 ± 7000 1 ± 0 1.1e-05 ± 8e-06 14 ± 1
3.5.0 111000 ± 4000 1 ± 0 1.2e-05 ± 7e-06 14.0 ± 0.7
dev 111000 ± 2000 1 ± 0 1.2e-05 ± 5e-06 13.9 ± 0.3
Wall time [sampling interval: 10]

Wall time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 103000 ± 8000 0.57 ± 0.004 1.3e-05 ± 7e-06 15 ± 1
3.5.0 110000 ± 4000 0.574 ± 0.006 1.2e-05 ± 5e-06 14.1 ± 0.6
dev 107000 ± 6000 0.566 ± 0.003 1.3e-05 ± 7e-06 15 ± 1
Wall time [sampling interval: 100]

Wall time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 10000 ± 200 0.004 ± 0.001 3e-05 ± 5e-05 18 ± 1
3.5.0 10100 ± 400 0.0037 ± 0.0005 6e-05 ± 4e-05 18.2 ± 0.4
dev 10300 ± 500 0.0036 ± 0.001 6e-05 ± 4e-05 17.5 ± 0.8
Wall time [sampling interval: 1000]

Wall time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 1770 ± 20 0.0005 ± 0.0006 0.0001 ± 0.0001 31 ± 3
3.5.0 1780 ± 20 0.0005 ± 0.0006 3e-05 ± 0.0001 29 ± 2
dev 1770 ± 20 0.0005 ± 0.0005 0.0001 ± 0.0002 30 ± 2
CPU time [sampling interval: 1]

CPU time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 44000 ± 4000 1 ± 0 2e-05 ± 2e-05 27 ± 1
3.5.0 45000 ± 2000 1 ± 0 3e-05 ± 1e-05 26.4 ± 0.5
dev 44000 ± 4000 1 ± 0 2e-05 ± 1e-05 26 ± 1
CPU time [sampling interval: 10]

CPU time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 46000 ± 2000 0.997 ± 0.005 2.2e-05 ± 8e-06 26.1 ± 0.7
3.5.0 43000 ± 6000 0.997 ± 0.004 2.3e-05 ± 8e-06 27 ± 2
dev 46000 ± 4000 0.997 ± 0.004 2e-05 ± 2e-05 26 ± 1
CPU time [sampling interval: 100]

CPU time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 4800 ± 100 0.01 ± 0.002 0.00012 ± 6e-05 32 ± 2
3.5.0 4800 ± 100 0.009 ± 0.002 0.0001 ± 0.0002 32 ± 1
dev 4870 ± 100 0.0085 ± 0.0009 0.0001 ± 0.0001 31.9 ± 0.7
CPU time [sampling interval: 1000]

CPU time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 906 ± 3 0.002 ± 0.001 0.0001 ± 0.0003 56 ± 4
3.5.0 908 ± 9 0.0007 ± 0.0008 0.0001 ± 0.0003 52 ± 3
dev 898 ± 5 0.0006 ± 0.0006 0.0004 ± 0.0004 52 ± 3
RSA keygen [sampling interval: 1]

RSA keygen [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 32000 ± 3000 1 ± 0 0.0003 ± 0.0003 30 ± 2
3.5.0 34000 ± 3000 1 ± 0 0.0002 ± 0.0002 28 ± 2
dev 35000 ± 2000 1 ± 0 0.0003 ± 0.0003 28 ± 1
RSA keygen [sampling interval: 10]

RSA keygen [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 34000 ± 2000 0.989 ± 0.008 0.0002 ± 0.0002 28 ± 2
3.5.0 33500 ± 900 0.987 ± 0.01 0.0003 ± 0.0002 28.9 ± 0.9
dev 33000 ± 1000 0.987 ± 0.007 0.0004 ± 0.0003 29 ± 2
RSA keygen [sampling interval: 100]

RSA keygen [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 6110 ± 70 0.019 ± 0.007 0.0006 ± 0.0005 37 ± 2
3.5.0 6070 ± 70 0.02 ± 0.01 0.0005 ± 0.0004 39 ± 3
dev 6140 ± 50 0.02 ± 0.009 0.001 ± 0.001 38 ± 2
RSA keygen [sampling interval: 1000]

RSA keygen [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 899 ± 5 0.0001 ± 0.0003 0.002 ± 0.003 64 ± 5
3.5.0 902 ± 5 0.001 ± 0.001 0.002 ± 0.002 61 ± 4
dev 900 ± 3 0.0004 ± 0.0008 0.001 ± 0.002 58 ± 3
Full metrics [sampling interval: 1]

Full metrics [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 42000 ± 3000 1 ± 0 3e-05 ± 2e-05 37 ± 3
3.5.0 41000 ± 4000 1 ± 0 2.3e-05 ± 9e-06 38 ± 5
dev 42000 ± 2000 1 ± 0 3e-05 ± 2e-05 36 ± 2
Full metrics [sampling interval: 10]

Full metrics [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 42000 ± 3000 0.999999 ± 4e-06 3e-05 ± 1e-05 36 ± 3
3.5.0 43000 ± 2000 0.999999 ± 4e-06 3.2e-05 ± 8e-06 35 ± 2
dev 43000 ± 4000 1.0 ± 1e-05 2e-05 ± 2e-05 36 ± 4
Full metrics [sampling interval: 100]

Full metrics [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 8700 ± 300 0.028 ± 0.004 0.0001 ± 8e-05 48 ± 2
3.5.0 8400 ± 200 0.03 ± 0.004 0.0001 ± 6e-05 49 ± 2
dev 8700 ± 300 0.026 ± 0.004 8e-05 ± 6e-05 46 ± 3
Full metrics [sampling interval: 1000]

Full metrics [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 1780 ± 30 0.0006 ± 0.0005 3e-05 ± 0.0001 68 ± 3
3.5.0 1770 ± 20 0.0005 ± 0.0004 0.0001 ± 0.0002 67 ± 3
dev 1760 ± 30 0.001 ± 0.0007 0.0001 ± 0.0002 69 ± 3
Multiprocess wall time [sampling interval: 1]

Multiprocess wall time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 2100 ± 200 1 ± 0 0.00018 ± 6e-05 320 ± 20
3.5.0 2700 ± 100 1 ± 0 0.0002 ± 0.0003 350 ± 10
dev 2600 ± 200 1 ± 0 0.0002 ± 0.0003 360 ± 30
Multiprocess wall time [sampling interval: 10]

Multiprocess wall time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 2100 ± 200 1.0 ± 1e-05 0.0003 ± 0.0003 310 ± 20
3.5.0 2700 ± 200 1 ± 0 0.00014 ± 6e-05 350 ± 30
dev 2700 ± 300 0.999998 ± 7e-06 0.0001 ± 5e-05 350 ± 40
Multiprocess wall time [sampling interval: 100]

Multiprocess wall time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 1900 ± 200 0.06 ± 0.03 0.00014 ± 6e-05 330 ± 10
3.5.0 2500 ± 100 0.06 ± 0.03 0.00012 ± 4e-05 370 ± 20
dev 2600 ± 100 0.07 ± 0.02 8e-05 ± 5e-05 360 ± 20
Multiprocess wall time [sampling interval: 1000]

Multiprocess wall time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 1700 ± 100 0.013 ± 0.004 3e-05 ± 3e-05 120 ± 20
3.5.0 2400 ± 100 0.017 ± 0.004 3e-05 ± 2e-05 130 ± 20
dev 2300 ± 100 0.018 ± 0.005 1e-05 ± 2e-05 130 ± 30

Benchmark Summary

Comparison of dev against 3.5.0.

The following scenarios show a statistically significant difference in performance between the two versions.

Sample Rate Saturation Error Rate Sampling Speed
Wall time [sampling interval: 10] 🟡 🟢 🟡 🟡
Wall time [sampling interval: 100] 🟡 🟡 🟡 🟢
CPU time [sampling interval: 1000] 🔴 🟡 🟡 🟡
RSA keygen [sampling interval: 100] 🟢 🟡 🟡 🟡
Full metrics [sampling interval: 100] 🟢 🟢 🟡 🟢

@P403n1x87 P403n1x87 merged commit fbec16f into devel Jun 4, 2023
@P403n1x87 P403n1x87 deleted the ci/cppcheck-from-sources branch June 4, 2023 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ci/cd
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@P403n1x87